Methods and Systems for Refining Text Color in a Digital Image

ABSTRACT

Aspects of the present invention relate to systems and methods for refining text color in a digital image.

FIELD OF THE INVENTION

Embodiments of the present invention comprise methods and systems forrefining text color in a digital image.

BACKGROUND

The content of a digital image may have considerable impact on thecompression of the digital image, both in terms of compressionefficiency and compression artifacts. Pictorial regions in an image maynot be efficiently compressed using compression algorithms designed forthe compression of text. Similarly, text images may not be efficientlycompressed using compression algorithms that are designed and optimizedfor pictorial content. Not only may compression efficiency be affectedwhen a compression algorithm designed for one type of image content isused on a different type of image content, but the decoded image mayexhibit visible compression artifacts.

Additionally, image enhancement algorithms designed to sharpen text, ifapplied to pictorial image content, may produce visually annoyingartifacts in some areas of the pictorial content. In particular,pictorial regions containing strong edges may be affected. Whilesmoothing operations may enhance a natural image, the smoothing of textregions is seldom desirable.

Copiers, scanners and other imaging devices may use text segmentationwhen performing content-specific processing and compression on document,and other digital, images. Exemplary content-specific processing maycomprise differential filtering and color enhancement. Exemplarycontent-specific compression may comprise layered compression schemes,where the contents of a document image are segmented into ahigh-resolution foreground layer and a lower resolution background.

Detection of text in digital images may be used so thatcontent-type-specific image enhancement methods may be applied to theappropriate regions in a digital image. The detection of regions of aparticular content type in a digital image may improve compressionefficiency, reduce compression artifacts, and improve image quality whenused in conjunction with a compression algorithm or image enhancementalgorithm designed for the particular type of content.

SUMMARY

Some embodiments of the present invention comprise methods and systemsfor deriving a refinement color value for a pixel based on neighboringpixel values.

The foregoing and other objectives, features, and advantages of theinvention will be more readily understood upon consideration of thefollowing detailed description of the invention taken in conjunctionwith the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL DRAWINGS

FIG. 1 is a drawing showing an examination window comprising text pixelsand non-text pixels;

FIG. 2. is a chart showing embodiments of the present inventioncomprising derivation of a replacement color value from color valuesclassified as similar to the color value of an examination pixel;

FIG. 3 is a chart showing embodiments of the present inventioncomprising derivation of a replacement color value from color valuesclassified as similar to the color value of an examination pixel when asufficient number of pixels may be classified as similar to the colorvalue of the examination pixel;

FIG. 4 is a chart showing embodiments of the present inventioncomprising derivation of a replacement color value from color valuesclassified as dissimilar to a background-color estimate;

FIG. 5 is a chart showing embodiments of the present inventioncomprising removing a pixel of interest from a text map when the colorvalue of the pixel of interest is similar to a background-colorestimate; and

FIG. 6 is a chart showing embodiments of the present inventioncomprising derivation of a replacement color value from color valuesclassified as dissimilar to a background-color estimate.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the present invention will be best understood byreference to the drawings, wherein like parts are designated by likenumerals throughout. The figures listed above are expressly incorporatedas part of this detailed description.

It will be readily understood that the components of the presentinvention, as generally described and illustrated in the figures herein,could be arranged and designed in a wide variety of differentconfigurations. Thus, the following more detailed description of theembodiments of the methods and systems of the present invention is notintended to limit the scope of the invention but it is merelyrepresentative of the presently preferred embodiments of the invention.

Elements of embodiments of the present invention may be embodied inhardware, firmware and/or software. While exemplary embodiments revealedherein may only describe one of these forms, it is to be understood thatone skilled in the art would be able to effectuate these elements in anyof these forms while resting within the scope of the present invention.

The content of a digital image may have considerable impact on thecompression of the digital image, both in terms of compressionefficiency and compression artifacts. Pictorial regions in an image maynot be efficiently compressed using compression algorithms designed forthe compression of text. Similarly, text images may not be efficientlycompressed using compression algorithms that are designed and optimizedfor pictorial content. Not only may compression efficiency be affectedwhen a compression algorithm designed for one type of image content isused on a different type of image content, but the decoded image mayexhibit visible compression artifacts.

Additionally, image enhancement algorithms designed to sharpen text, ifapplied to pictorial image content, may produce visually annoyingartifacts in some areas of the pictorial content. In particular,pictorial regions containing strong edges may be affected. Whilesmoothing operations may enhance a natural image, the smoothing of textregions is seldom desirable.

Copiers, scanners and other imaging devices may use text segmentationwhen performing content-specific processing and compression on document,and other digital, images. Exemplary content-specific processing maycomprise differential filtering and color enhancement. Exemplarycontent-specific compression may comprise layered compression schemes,where the contents of a document image are segmented into ahigh-resolution foreground layer and a lower resolution background.

Detection of text in digital images may be used so thatcontent-type-specific image enhancement methods may be applied to theappropriate regions in a digital image. The detection of regions of aparticular content type in a digital image may improve compressionefficiency, reduce compression artifacts, and improve image quality whenused in conjunction with a compression algorithm or image enhancementalgorithm designed for the particular type of content.

Text segmentation algorithms may identify textual content in an image. Atext map may indicate which pixels in an image may correspond to textpixels. In some embodiments of the present invention, a text map may bea binary image, array or other data structure, in which the binary valueof a pixel at a location in the text map may indicate whether or not thecorresponding pixel in the image data may be part of a text character.The color value of the corresponding pixel in the image data may bereferred to as the text-pixel color value or the non-text-pixel colorvalue according to the binary value.

In alternative embodiments of the present invention, a text map may be amulti-valued image, array or other data structure, in which the value ofa pixel at a location in the text map may indicate a certainty measurefor the corresponding pixel in the image data. The certainty measure mayindicate the membership value of the pixel in the text class. A textpixel may refer to a pixel with a text-map value in particularrelationship to a threshold. The color value of the corresponding pixelin the image data may be referred to as the text-pixel color value.

Associated with a text pixel may be a text color. Noise and otherartifacts may be introduced into a document image during a scanningprocessing which may result in color variations in connected pixels thatconstitute a single character, as well as in neighboring characterswithin a text block. Some embodiments of the present invention comprisemethods and systems for reducing such fluctuations in text color.

In some embodiments of the present invention, a plurality of text-mappixels in a window centered around a pixel of interest may be examined.FIG. 1 shows an exemplary window 2 in a text map with text-map pixelscentered around a center pixel 4. The center pixel 4 may be referred toas a pixel of interest or the examination pixel. For a binary text-map,a pixel in the window 2 may have a first value indicating that the pixelbelongs to the text class. For example, a shaded pixel (of which 4 and 5are exemplary), may indicate that the pixel is a text pixel. While anun-shaded pixel (of which 6 is exemplary), may indicate that the pixelis not a text pixel.

In some embodiments of the present invention, a refinement color value,also considered a replacement color value, for a text pixel of interestmay be determined by examining the pixels in an examination windowsurrounding the text pixel of interest. In these embodiments, describedin relation to FIG. 2, a color value representing a first color classmay be selected 10. Each text pixel in the examination window may beclassified 11 into the first class or a second class based on thesimilarity between the color value of the text pixel and therepresentative color value for the first class. The refinement colorvalue for the text pixel of interest may be derived 12 based on thecolor values of the pixels in each of the two classes.

Pixel color values may comprise a color representation in any colorspace including, but not limited to, RGB, sRGB, CMYK, YUV, YIQ, YCbCr,YPbPr, HSV, HSL, Lab and L*a*b*.

In some embodiments of the present invention, color similarity may bemeasured according to:

$\frac{{\alpha {{A_{i} - A_{rep}}}} + {\beta {{B_{i} - B_{rep}}}} + {\gamma {{C_{i} - C_{rep}}}}}{\alpha + \beta + \gamma},$

where (A_(i), B_(i), C_(i)) may be the color-component values for theith text pixel in the examination window, (A_(rep), B_(rep), C_(rep))may be the color-component values for the representative color value ofthe first class and (α, β, γ) may be weights associated with each colorcomponent. The contribution of each color component to the similaritymeasure may be controlled by the weights (α, β, γ).

In alternative embodiments of the present invention, color similaritymay be measured using a distance measurement between the text pixelcolor and the representative color. Exemplary color-distance measuresmay comprise an L₁ norm, an L₂ norm, a 2-dimensional city block distancemeasure between the chroma components of a luma-chroma-chroma colorspace representation, a 3-dimensional city block distance measurebetween the components of a 3-dimensional color space representation, aEuclidean distance measure, a weighted 2-dimensional city block distancemeasure between the chroma components of a luma-chroma-chroma colorspace representation, a weighted 3-dimensional city clock distancebetween the components of a 3-dimensional color space representation andother well-known-in-the-art distance measures.

Some embodiments of the present invention may be described in relationto FIG. 3. In these embodiments, the representative color value for thefirst class may be set 30 to the color value of the text pixel ofinterest. Text pixels classified 31 as belonging to the first class maycomprise text pixels with color values substantially similar to thecolor value of the text pixel of interest. Text pixels classified 31 asbelonging to the second class may comprise text pixels with color valuessubstantially dissimilar to the color value of the text pixel ofinterest. Color classification 31 may comprise comparing the similaritymeasure to a predetermined threshold. The membership count in the eachof the two classes may be determined 32. Based on the membership counts33, the replacement color value for the text pixel of interest may bedetermined from the color values of the text pixels in the first class37 or the second class 35.

In some embodiments of the present invention, the replacement colorvalue may be derived from the color values of the pixels in the firstclass 37 when the condition (C1<Tsim and C1+C2≧Ttot), where C1 and C2may be the membership counts of the first class and the second class,respectively, and Tsim may be a first threshold and Ttot may be a secondthreshold, may be satisfied 36. The replacement color value may bederived from the color values of the pixels in the second class 35 whenthis condition is not satisfied 34.

In some embodiments of the present invention, the replacement colorvalue, denoted (ω_(A), ω_(B), ω_(C)), may be derived from the colorvalues of the pixels in the first class 37 according to:

${\omega_{A} = {\frac{1}{C\; 1}{\sum\limits_{i \in {{class}\; 1}}A_{i}}}},{\omega_{B} = {\frac{1}{C\; 1}{\sum\limits_{i \in {{class}\; 1}}B_{i}}}},{\omega_{C} = {\frac{1}{C\; 1}{\sum\limits_{i \in {{class}\; 1}}{C_{i}.}}}}$

The replace color value may be derived from the color values of thepixels in the second class 35 according to:

${\omega_{A} = {\frac{1}{C\; 2}{\sum\limits_{i \in {{class}\; 2}}A_{i}}}},{\omega_{B} = {\frac{1}{C\; 2}{\sum\limits_{i \in {{class}\; 2}}B_{i}}}},{\omega_{C} = {\frac{1}{C\; 2}{\sum\limits_{i \in {{class}\; 2}}{C_{i}.}}}}$

In alternative embodiments, the replacement color value may be the classmedian color value.

In alternative embodiments of the present invention, the color value forall text pixels within the examination window may be updated to thereplacement color value. In still alternative embodiments, the colorvalue for all text pixels within the examination window and within thefirst class may be updated to the replacement color value.

In some embodiments of the present invention, a local-background-colorestimate may be used in the color refinement process. These embodimentsmay be described in relation to FIG. 4. In some embodiments of thepresent invention, a refinement color value, also considered areplacement color value, for a text pixel of interest may be determinedby examining the pixels in an examination window surrounding the textpixel of interest. In these embodiments, a color value representing afirst color class may be set to a local-background-color estimate 40.The text pixels in an examination window may be classified into thefirst class and a second class 41. The classification 41 may be based onthe similarity of the text pixel color value to the representative colorvalue. Similarity may be determined by methods described above. Thereplacement, or refinement, color value may be derived 42 from the colorvalues text pixels classified into the second class. In someembodiments, the color value of the text pixel of interest around whichthe examination window is centered may be updated to the replacementcolor value. In alternative embodiments, the color value of all textpixels within the examination window may be updated to the replacementcolor value. In still alternative embodiments, the color value for alltext pixels within the examination window that are also in the secondclass may be updated to the replacement color value.

In some embodiments of the present invention, if the color value of thetext pixel of interest is substantially similar to thelocal-background-color estimate, then the text pixel may be removed fromthe text map. These embodiments may be described in relation to FIG. 5.A local-background-color estimate may be determined 50. Thelocal-background-color estimate may be provided as the result offoreground/background, or other processing, in a processing pipeline. Inthe absence of a preprocessing module that explicitly provides thelocal-background-color estimate, the local-background-color estimate maybe determined from the color values of the pixels within the examinationwindow. In some embodiments, the local-background-color estimate maycomprise the mean, median, trimmed mean, or other combination, of thecolor values of the non-text pixels. The local-background-color estimatemay be propagated from examination window to examination window, and thepropagated color value used if all of the pixels within an examinationwindow are text pixels. The examination pixel color value may becompared to the local-background-color estimate 51, and if theexamination pixel color value is substantially similar to thelocal-background-color estimate 52, then the examination pixel may beremoved from the text map 53. If the examination pixel color value isnot substantially similar to the local-background-color estimate 54,then the local-background-color estimate may be selected 55 as therepresentative color value of a first color class. The text pixels inthe examination window may be classified 56 into the first class and asecond class. The similarity of the local-background-color estimate anda text pixel color value may be used to classify 56 the text pixel. Thereplacement color value may be derived from the color values of the textpixels classified in the second class 57. The replacement color valuemay be the mean, trimmed mean, median, or other combination of the colorvalues of the text pixels in the second class. In alternativeembodiments, the replacement color value may be the color value from thesecond class pixels that is maximally dissimilar to thebackground-color-estimate.

In alternative embodiments of the present invention, the color value forall text pixels within the examination window may be updated to thereplacement color value. In alternative embodiments, the color value forall text pixels within the examination window that are also in thesecond class may be updated to the replacement color value.

In some embodiments of the present invention, a refinement color value,also considered a replacement color value, for a pixel of interest maybe determined by examining the pixels in an examination windowsurrounding the pixel of interest. In these embodiments, all of thepixels in an examination window may be examined.

Some embodiments of the present invention may be described in relationto FIG. 6. In these embodiments, the representative color value for thefirst class may be set 60 to the color value of the pixel of interest.All pixels in the examination window may be examined, and pixels in theexamination window classified 61 as belonging to the first class maycomprise pixels with color values substantially similar to the colorvalue of the pixel of interest. While pixels in the examination windowclassified 61 as belonging to the second class may comprise pixels withcolor values substantially dissimilar to the color value of the pixel ofinterest. Color classification 61 may comprise comparing the similaritymeasure to a predetermined threshold. The membership count in the eachof the two classes may be determined 62. Based on the membership counts63, the replacement color value for the pixel of interest may bedetermined from the color values of the pixels in the first class 67 orthe second class 65.

In some embodiments of the present invention, the replacement colorvalue may be derived from the color values of the pixels in the firstclass 67 when the condition (C1<Tsim and C1+C2≧Ttot), where C1 and C2may be the membership counts of the first class and the second class,respectively, and Tsim may be a first threshold and Ttot may be a secondthreshold, may be satisfied 66. The replacement color value may bederived from the color values of the pixels in the second class 65 whenthis condition is not satisfied 64.

In some embodiments of the present invention, the replacement colorvalue, (ω_(A), ω_(B), ω_(C)), may be derived from the color values ofthe pixels in the first class 67 according to:

${\omega_{A} = {\frac{1}{C\; 1}{\sum\limits_{i \in {{class}\; 1}}A_{i}}}},{\omega_{B} = {\frac{1}{C\; 1}{\sum\limits_{i \in {{class}\; 1}}B_{i}}}},{\omega_{C} = {\frac{1}{C\; 1}{\sum\limits_{i \in {{class}\; 1}}{C_{i}.}}}}$

The replace color value may be derived from the color values of thepixels in the second class 65 according to:

${\omega_{A} = {\frac{1}{C\; 2}{\sum\limits_{i \in {{class}\; 2}}A_{i}}}},{\omega_{B} = {\frac{1}{C\; 2}{\sum\limits_{i \in {{class}\; 2}}B_{i}}}},{\omega_{C} = {\frac{1}{C\; 2}{\sum\limits_{i \in {{class}\; 2}}{C_{i}.}}}}$

In alternative embodiments, the replacement color value may the classmedian color value.

In alternative embodiments of the present invention, the color value forall pixels within the first class in the examination window may beupdated to the replacement color value.

In some embodiments of the present invention, a text pixel of interestmay be removed from the text map if the number of text pixels within anexamination window is below a threshold.

In some embodiments of the present invention, a sliding examinationwindow may be used wherein the center text pixel color value may berefined. In alternative embodiments of the present invention,non-overlapping examination windows may be used wherein all of the colorvalue of all of the text pixels within the examination window may berefined. In still alternative embodiments, partially overlappingexamination windows may be used wherein only the color value of the textpixels within the non-overlapping part of the block may be refined.

In some embodiments of the present invention, a replacement value may befurther refined based on a selection of the most recently determinedrefinement values. In alternative embodiments, the further refinementmay be based on the refinement values in a causal neighborhood of thepixel of interest.

The terms and expressions which have been employed in the foregoingspecification are used therein as terms of description and not oflimitation, and there is no intention in the use of such terms andexpressions of excluding equivalence of the features shown and describedor portions thereof, it being recognized that the scope of the inventionis defined and limited only by the claims which follow.

1. A method for determining a refinement color value for a pixel ofinterest in a digital image, said method comprising: a) from a digitalimage comprising a first plurality of pixels, wherein each pixel in saidfirst plurality of pixels comprises an associated color value, selectinga pixel of interest from said first plurality of pixels, said pixel ofinterest comprising a pixel-of-interest color value; b) separating saidfirst plurality of pixels into a second plurality of pixels and a thirdplurality of pixels based on said pixel-of-interest color value; c)calculating a refinement color value based on said second plurality ofpixels when said second plurality of pixels meets a quantity threshold;d) calculating said refinement color value based on said third pluralityof pixels when said second plurality of pixels does not meet saidquantity threshold; and e) assigning said refinement color value to saidpixel of interest.
 2. A method according to claim 1, wherein said pixelof interest is the center pixel in an examination window comprising saidfirst plurality of pixels.
 3. A method according to claim 1, whereinsaid first plurality of pixels are text pixels.
 4. A method according toclaim 3 further comprising assigning said refinement color value to saidfirst plurality of pixels.
 5. A method according to claim 1, whereinsaid second plurality of pixels comprises pixels from said firstplurality of pixels with associated color values substantially similarto said pixel-of-interest color value.
 6. A method according to claim 5further comprising assigning said refinement color value to said secondplurality of pixels.
 7. A method according to claim 1, wherein saidcalculating a refinement color value based on said second plurality ofpixels comprises a computation selected from the group consisting of amean of the associated color values of the pixels in said secondplurality of pixels, a median of the associated color values of thepixels in said second plurality of pixels, a weighted average of theassociated color values of the pixels in said second plurality of pixelsand a trimmed-mean of the associated color values of the pixels in saidsecond plurality of pixels.
 8. A method according to claim 1, whereinsaid calculating a refinement color value based on said third pluralityof pixels comprises a computation selected from the group consisting ofa mean of the associated color values of the pixels in said thirdplurality of pixels, a median of the associated color values of thepixels in said third plurality of pixels, a weighted average of theassociated color values of the pixels in said second plurality of pixelsand a trimmed-mean of the associated color values of the pixels in saidthird plurality of pixels.
 9. A method according to claim 1, whereinsaid separating said first plurality of pixels into said secondplurality of pixels and said third plurality of pixels based on saidpixel-of-interest color value comprises: a) calculating a similaritymeasure between said pixel-of-interest color value and said associatedcolor value of a second pixel from said first plurality of pixels; andb) associating said second pixel with said second plurality of pixelswhen said similarity measure meets a similarity criterion.
 10. A methodaccording to claim 9, wherein said similarity measure is a distancemeasure selected from the group consisting of L₁ norm, L₂ norm,Euclidean distance, city-block distance, weighted city-block distanceand weighted Euclidean distance.
 11. A system for determining arefinement color value for a pixel of interest in a digital image, saidsystem comprising: a) a pixel-of-interest selector for, from a digitalimage comprising a first plurality of pixels, wherein each pixel in saidfirst plurality of pixels comprises an associated color value, selectinga pixel of interest from said first plurality of pixels, said pixel ofinterest comprising a pixel-of-interest color value; b) a classifier forseparating said first plurality of pixels into a second plurality ofpixels and a third plurality of pixels based on said pixel-of-interestcolor value; c) a refinement color calculator for: i) calculating arefinement color value based on said second plurality of pixels whensaid second plurality of pixels meets a quantity threshold; and ii)calculating said refinement color value based on said third plurality ofpixels when said second plurality of pixels does not meet said quantitythreshold; and d) color-value assigner for assigning said refinementcolor value to said pixel of interest.
 12. A method for determining acolor value for a pixel of interest in a digital image, said methodcomprising: a) from a digital image comprising a first plurality ofpixels, wherein each pixel in said first plurality of pixels comprisesan associated color value, selecting a pixel of interest from said firstplurality of pixels, said pixel of interest comprising apixel-of-interest color value; b) identifying from said first pluralityof pixels a second plurality of pixels, wherein said color valuesassociated with said second plurality of pixels are substantiallydissimilar to a background-color estimate; c) calculating a refinementcolor value based on said second plurality of pixels; and d) assigningsaid refinement color value to said pixel of interest.
 13. A methodaccording to claim 12 further comprising calculating saidbackground-color estimate.
 14. A method according to claim 12 furthercomprising calculating said refinement color value based on said secondplurality of pixels when said second plurality of pixels meets aquantity threshold.
 15. A method according to claim 12 furthercomprising removing said pixel of interest from an associated text mapwhen said pixel of interest color value is substantially similar to saidbackground-color estimate.
 16. A method according to claim 12, whereinsaid pixel of interest is the center pixel in an examination windowcomprising said first plurality of pixels.
 17. A method according toclaim 12, wherein said first plurality of pixels are text pixels.
 18. Amethod according to claim 17 further comprising assigning saidrefinement color value to said first plurality of pixels.
 19. A methodaccording to claim 12 further comprising assigning said refinement colorvalue to said second plurality of pixels.
 20. A method according toclaim 12, wherein said calculating a refinement color value based onsaid second plurality of pixels comprises a computation selected fromthe group consisting of a mean of the associated color values of thepixels in said second plurality of pixels, a median of the associatedcolor values of the pixels in said second plurality of pixels, aweighted average of the associated color values of the pixels in saidsecond plurality of pixels and a trimmed-mean of the associated colorvalues of the pixels in said second plurality of pixels.
 21. A methodaccording to claim 12, wherein said refinement color value comprises amaximally dissimilar color value from said associated color values ofsaid second plurality of pixels relative to said background-colorestimate.
 22. A method according to claim 12, wherein said identifyingfrom said first plurality of pixels a second plurality of pixelscomprises: a) calculating a similarity measure between saidbackground-color estimate and said associated color value of a secondpixel from said first plurality of pixels; and b) associating saidsecond pixel with said second plurality of pixels when said distancemeasure meets a dissimilarity criterion.
 23. A method according to claim22, wherein said similarity measure is a distance measure selected fromthe group consisting of L₁ norm, L₂ norm, Euclidean distance, city-blockdistance, weighted city-block distance and weighted Euclidean distance.